CLT&MEP 
Maximum entropy approach to central limit distributions of correlated variables 
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Hilhorst and Schehr recently presented a straight forward computation of limit distributions of 
sufficiently correlated random numbers [1(. Here we present the analytical form of entropy which - 
under the maximum entropy principle (with ordinary constraints)- provides these limit distributions. 
These distributions are not g-Gaussians and can not be obtained with Tsallis entropy. 

PACS numbers: 05.70.-a 05.90.+m 05.20.-y 



INTRODUCTION 

Classical statistical mechanics is tightly related to the 
central limit theorem (CLT). For example if one inter- 
prets the velocity of an ideal-gas particle as the result 
of N random collisions with other particles, the velocity 
distribution of particles corresponds to a iV-fold convo- 
lution some distribution of momenta exchanges. For any 
such distribution, as long as it is centered, stable and 
has a second moment, the central limit theorem imme- 
diately guarantees Maxwell-Boltzmann distributions for 
N — > oo. Alternative to this mathematical approach the 
same distribution can be derived from a physical princi- 
ple, where Boltzmanns P-function (entropy) gets max- 
imized under the constraint that average kinetic energy 
is proportional to temperature, ksT. This principle is 
referred to as the maximum entropy principle (MEP). 
These results are trivial for the ideal-gas, or equivalently, 
for independent random numbers. However, as soon as 
correlations come into play things become more involved 
on both sides: CLTs for correlated random numbers have 
regained strong interest [2|, |3J, 1J] • Recently a general and 
transparent method of obtaining limit distributions of 
correlated random numbers was reported for a wide class 
of processes [l|. In the context of the MEP there ba- 
sically exist three methods to arrive at non-Boltzmann 
(non-Gaussian) distributions (with reasonably many con- 
straints). The first is associated with Tsallis entropy 
which produces g-exponentials or g-Gaussians under en- 
tropy maximization [5j, the second is based on the so- 
called K-logarithm [6] leading to a three-parameter class 
of distribution functions. Recently a method was intro- 
duced to constructively design entropy functionals which 

under maximization under ordinary constraints - pro- 
duce any plausible distribution function [7j. This was 
later shown to be a thermodynamically relevant entropy 
[8| . This entropy is a generalization of previous general- 
izations of Boltzmann-Gibbs entropies therefore we call 
it generalized- generalized entropy, S gg . 

In this paper we show that CLTs and the MEP can 



be brought into a consistent framework for correlated 
variables. We start by reviewing the CLT for sums of 
correlated random numbers, following [l|, and the MEP 
derivation for arbitrary distribution functions following, 
[7f . We then give the explicit form of the entropy leading 
to Hilhorst and Schehr's limit distributions. 



Limit theorems of correlated random numbers 

The idea in [l| is to consider a totally symmetric cor- 
related Gaussian Appoint process 



Pn(z) = 



H z - M ~ lz ) 



y/(2n) N det(M) 



(1) 



with z = (zi, . . . , zjy) and M the covariance matrix. 
This stochastic process is used as a reference process for 
some other totally symmetric Appoint distribution Pjy (w) 
which is related to Pv(z) by a simple transformation of 
variables itj = h(zi), for alH = 1, . . . , N. Total symmetry 
dictates the form the covariance matrix 



Mij = Sij + p(l — 6ij) 



(2) 



for p G (0, 1] with the inverse, M-- = aSij — 6(1 — Sij), 

where a = ( 1 _ p ) (1+ ( A r_ 1 ) p ) , and b = ( 1 _ p )( 1+ (j V -i)p)- 
Straight forward calculation yields that the marginal 
probability is a Gaussian with unit variance, 



p Gauss (zi) = J 



dz 2 ...dz N P N (zi,z 2 ,...,z N ) . (3) 



This allows to construct the set of variables Ui from Zi 
via the transformation of variables 



r du' p^u') = r dz' pp auss { z ') 

Jo Jo 



(4) 



Pi being the one-point distribution of the u variables. 
Consequently, there is a unique function h, such that 



h(zi) 



(5) 



The distribution of the average of the u variables, 

JV 

N 



1 

u = — > Uj 



(6) 



i=l 



is thus found in terms of an integration over all Zi 

v(u) = JdzP N (z)6L~±j2 h ( z >n * 



(7) 



where dz = dz\ . . . dzjsr- After some calculation one ar- 
rives at the general result [lj , 



V(u) = 



1-P 



\k' (v*(u))\ 1 exp 



1-Pr 



K 



2tt/> / ' v w/l V 2p 

where i/» is defined as the zero of the function 
1 



■ («)] S 



(8) 



k(v)=u -= \ dw e~^~h ((w + v)\Jl - p) , (9) 

and fc'(x) = d/dx k(x). For symmetric one-point distri- 
butions Pi, ft. and v* are both antisymmetric. Moreover 
it is seen that v' r {u) — —k'(v*(u))~ 1 > 0, such that 



^ ) = ^KvV"* ( *° 



(10) 



Generalized-generalized entropy 

The presently most general form of entropy that is con- 
sistent with the maximum entropy condition reads in di- 
mensionless notation [7|, 



with 



S gg [P]=-J2 p ^)HP(zi))+r,[P] 



r)[P] =^2 I dxx — r-— ■+ '■ 



d:X 



which can be rewritten as 



, Jo 



P(*i) 



dx A(x) + c 



(11) 



(12) 



(13) 



In these equations, P{zi) is a normalized distribution 
function of some parameter set z |ll| In physics this could 
be e.g. energy or velocity. A(x) is integrable in each in- 
terval [0, P(zi)]. It can be seen as a generalized logarithm 
satisfying A(x) < and dA(x)/dx > for < x < 1, and 
xA(x) — > (x —* 0+), making S gg [P] non-negative and 
concave, c is a constant, which ensures that S gg [Pa] = 
for a completely ordered state, i.e. c = — L dxA(x). 



A generalized maximum entropy method, given the ex- 
istence of some arbitrary stationary distribution function, 
P(zi), is formulated as 



SG 
SP(zi) 



\p=p 







with 



G — Sgg[P] — 



Z^ 



P(Zi)-l}-0- 



Z^ 



(14) 



f(zi)P{zi) - U 



(15) 
where a and f3 are Lagrange multipliers, and U denotes 
the expectation of function /, which depending on the 
problem may be a particular moment of z. The station- 
ary solution to this problem is given by 



P(z i ) = £(-a-f3f(z l )) 



(16) 



where £{x) is a generalized exponential, which is the in- 
verse function of A(x): £(A(x)) = A(£(x)) = x. In other 
words, A(x) from Eq. (1 1 1 1) is chosen as the inverse func- 
tion of the stationary (observed) distribution function. 

The form of entropy in Eq. (fT3"|) is enforced by the 
maximum entropy principle. Under variation the first 
term on the right hand side of Eq. (fTTj) yields -^PA{P) = 
A + P^§. The term P^ can neither get absorbed into 
the logarithmic terms, nor by the constants. As long 
as this term is present, the only solution to the MEP is 
A = In. The idea of the generalized-generalized entropy 
is to introduce r\ (which modifies entropy), such that the 
term Pj^ cancels out exactly under the variation, for 
details see [7| . It has been shown explicitly that the first 
and second laws of thermodynamics are robust under this 
generalization of entropy [8(. For the discussion below 
note that introducing a scaling factor £ in the argument, 



S gg [P\ 



PM 
E / ^ A (Cz) 



corresponds to distributions of the form 



1 



P(z) = -£(-a-0f(z)) 



(17) 



(18) 



C and a can be seen as two alternative normalization 
parameters. For instance the condition £ = 1 leads to a 
normalization of the distribution as discussed in [7|, |g|, 
while the condition a = leads to a normalization of P 
where £ plays the role of the partition function. 



ENTROPY FOR LIMIT DISTRIBUTIONS 

Let us now properly identify the distributions P and V 
from Eqs. dH]) and (fl"7| . First, note that the limit distri- 
butions in [l[ are centered and symmetric thus the first 
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FIG. 1: Top: Limit distributions al la Hilhorst-Schehr for p — 0.001 (left) 0.5 (middle), and 0.7 (right) (lines). Broken lines 
are g-Gaussians with a q from the reported best-fit value of q — 1 J ? n , lj. Bottom: Generalized logarithms necessary to 



i-p 



provide Hilhorst-Schehr distribution functions under the maximum entropy principle (lines). Broken lines are g-logarithms 
ln^a;) = (a; 1-5 — 1)/(1 — q) for the same q values. 



moment does not provide any information. We therefore 
choose / from Eq. (fT"5)) to be f(z) = z 2 . Second, the 
one point distribution P\ has fixed variance and so does 
V(u) in Eq. ([5]), whereas distributions obtained through 
the MEP Eq. (JT5J) scale with a function of the " inverse 
temperature" , /3. To take this into consideration for the 
identification of P and V we need a simple scale trans- 
formation u — \z, where A(/3) depends explicitly on (3. 
Consequently, V(u) — » XP(Xz) and 



XV(\z) 



J.(- 



f3z 2 



(19) 



This particular identification and the independence of 
Lagrange parameters requires the normalization condi- 
tion a — for the limit distribution Eq. (fT8|) . Fur- 
ther, to determine £ and A we use two conditions usually 
valid for generalized exponential functions, £(0) = 1 and 
£'{0) = 1. This leads to 



1 



P 



2np 



vl(0) and A 



fV0 



(20) 



2pi/(0) 



with 7 = K{0Y(1 _ p) _ vl ,, {0)p . 
tial can now be identified as 



The generalized exponen- 



£{x) 






■ exp 



2p 



^*(7v /= x)] ; 



(21) 



This uniquely defines £ on the domain (— oo, 0]. Finally, 
the generalized logarithm A is uniquely defined on the 
domain (0, 1] as the inverse function of £ and can be 
given explicitly for specific examples. 

An example 

The special case of a block function Pi(uj) = 1 for 
— h < u j < \ wa s discussed in [l| . This choice implies 



k'{v) = -'■ 



2(2 - p) 



k{v) 



u--eii{Kv) , v* = K _1 erf 1 (2u), (22) 



and the limit distribution Eq 
'2-p 



becomes 



V{u) = 



P 



exp 



2(1 ~ p) [crr 1 (2«)] 2 



(23) 

This block function has been used earlier [9(] where it 
was conjectured on numerical evidence that the limiting 
distribution would be a q-Gaussian. This is obviously 
ruled out by Eq. (|23[) . however, actual discrepancy is 
small, see Fig. [U For this example Eq. |2T|) becomes 



£{x) = exp (-(7T7 2 )- 1 [err 1 (2 7x /^)] 2 ) 



(24) 



where 7 = y/p/(2ir(l — p)). The associated generalized 
logarithm can be explicitly given on the domain (0, 1] 



A(x) 



(27) 1 erf I 7V— 7rlnx J 



(25) 



It is compared to g-logarithms in Fig. [l] the discrepancy 
is small but visible. 



DISCUSSION 

Historically it was hypothesized on numerical evidence 
9] that limit distributions of sums of correlated random 
numbers generated along the lines of Eq. ([3|) might be 
deeply related to g-Gaussians, thus lending fundamental 
support for g-entropy. In |l| it was shown that this is not 
the case. If g-entropy does not lead to the exact limit dis- 
tributions under the MEP, which form of entropy does? 
Here we constructively answer this question by using a 
recently proposed generalization of g-gcncralizcd entropy 
[7|, which is thermodynamically relevant [8J. Interest- 
ingly, a similar program has been carried out some time 
ago for Levy stable distributions, which result from a gen- 
eralized CLT where higher momenta do not exist. There 
it was shown that the corresponding entropy functional 
is Tsallis entropy [lOj, however under ^-constraints. 
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